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Abstract. Many combinatorial optimization problems entail a number of hierar- 
chically dependent optimization problems. An often used solution is to associate 
a suitably large cost with each individual optimization problem, such that the so- 
lution of the resulting aggregated optimization problem solves the original set of 
hierarchically dependent optimization problems. This paper starts by studying the 
package upgradeability problem in software distributions. Straightforward solu- 
tions based on Maximum Satisfiability (MaxSAT) and pseudo-Boolean (PB) op- 
timization are shown to be ineffective, and unlikely to scale for large problem in- 
stances. Afterwards, the package upgradeability problem is related to multilevel 
optimization. The paper then develops new algorithms for Boolean Multilevel 
Optimization (BMO) and highlights a large number of potential applications. The 
experimental results indicate that the proposed algorithms for BMO allow solving 
optimization problems that existing MaxSAT and PB solvers would otherwise be 
unable to solve. 



1 Introduction 

Many real problems require an optimal solution rather than any solution. Whereas deci- 
sion problems require a yes/no answer, optimization problems require the best solution, 
thus differentiating the possible solutions. In practice, there must be a classification 
scheme to determine how one solution compares with the others. Such classification 
may be seen as a way of establishing preferences that express cost or satisfaction. 

A special case of combinatorial optimization problems may require a set of opti- 
mization criteria to be observed, for which is possible to define a hierarchy of impor- 
tance. Suppose that instead of requiring a balance between price, horsepower and fuel 
consumption for choosing a new car, you have made a clear hierarchy in your mind: you 
have a strict limit on how much you can afford, then you will not consider a car with 
less than 150 horsepower and after that the less the fuel consumption the better. Not 
only you establish a hierarchy in your preferences, but also the preferences are defined 
in such a way that the set of potential solutions gets subsequently reduced. Such kind 
of problems are present not only in your daily life but also in many real applications. 



Clearly, the kind of problems we target can be encoded as a constraint optimiza- 
tion problem, making use of the available technology for dealing with preferences. 
Preference handling is one of the current hot topics in AI with active research lines 
in constraint satisfaction and optimization [24|. Broadly, preferences over constraints 
may be expressed quantitatively or qualitatively. For example, one may wish to fly in 
the afternoon or simply choose the less expensive flight of that day. Soft constraints 
model quantitative preferences by associating a level of satisfaction with each of the 
solutions |23|, whereas CP-nets model qualitative preferences by expressing preferen- 
tial dependencies with pairwise comparisons fl). Furthermore, preference-based search 
algorithms can be generalized to handle multi-criteria optimization [16|. 

A straightforward approach to solve a special case of a constraint optimization prob- 
lem, for which there is a total ranking of the criteria, would be to establish a lexico- 
graphic ordering over variables and domains, such that optimal solutions would come 
first in the search tree ifTSl . But this has the potential disadvantage of producing a thrash- 
ing behavior whenever assignments that are not supported by any solution are consid- 
ered, as a result of decisions made at the first nodes of the search tree [16,1. 

Maximum satisfiability (MaxSAT) naturally encodes a constraint optimization prob- 
lem over Boolean variables where constraints are encoded as clauses. A solution to the 
MaxSAT problem maximizes the number of satisfied clauses. Weights may also be as- 
sociated with clauses, in which case the sum of the weights of the satisfied clauses is 
to be maximized. The use of the weighted MaxSAT formalism allows to solve a set 
of hierarchically dependent optimization problems. Pseudo-Boolean (PB) optimization 
may also be used to solve this kind of problems, given that weighted MaxSAT problem 
instances can be translated to PB. Each clause is extended with a relaxation variable 
that is then included in the cost function, jointly with the respective weight. 

Boolean satisfiability (SAT) and PB have been extended in the past to handle pref- 
erences. For example, SAT-based planning has been extended to include conflicting 
preferences [14J, for each of which weights are associated, thus requiring the use of an 
objective function involving the preferences and their weights. The proposed solution 
modifies a SAT backtracking algorithm to search first for optimal plans by branching 
according to the partial order induced by the preferences. In addition, algorithms for 
dealing with multi-objective PB problems have been developed lfT9l . in contrast to tra- 
ditional algorithms that optimize a single linear function. 

This paper is organized as follows. The next section describes the problem of pack- 
age upgradeability in software systems. This problem comes from a real application and 
has been the drive for the algorithms being developed. Section [3]introduces multilevel 
optimization and relates it with a variety of problems. Afterwards, specific multilevel 
optimization algorithms are proposed, being based in MaxSAT and PB. Experimental 
results show the effectiveness of the new algorithms. Finally, the paper concludes. 

2 A Practical Example 

We have all been through a situation where the installation of a new piece of software 
turns out to be a nightmare. Not only you do not get the new computer program in- 
stalled, but also some other programs may eventually stop working properly. And this 



may also happen when you simply want to upgrade to a more recent version of a pro- 
gram that you have been using for some time. Although this seems to be a software 
engineering problem, behind the nightmare is a hard computational problem, and there- 
fore an intelligent solution is desirable. 

These kind of problems may occur because there are constraints between the differ- 
ent pieces of software (called packages). Although these constraints are expected to be 
handled in a consistent and efficient way, current software distributions are developed 
by distinct individuals. This is opposed to traditional systems which have a central- 
ized and closed development. Open systems also tend to be much more complex, and 
therefore some packages may become incompatible. In such circumstances, user prefer- 
ences should be taken into account. For example, you would rather prefer to have your 
old version of skype working than to have the latest version not working properly. 

The constraints associated with each package can be defined by a tuple {p, D, C), 
where p is the package, D are the dependencies of p, and C are the conflicts of p. D is 
a set of dependency clauses, each dependency clause being a disjunction of packages. 
C is a set of packages conflicting with p. 

Previous work has applied SAT-based tools to ensure the consistency of reposito- 
ries and installations as well as to upgrade consistently package installations. SAT-based 
tools have first been used to support distribution editors [20 1 . The developed tools are 
automatic and ensure completeness, which makes them more reUable than ad-hoc and 
manual tools. Recently, Max-SAT has been applied to solve the software package instal- 
lation problem from the user point of view [5 1. In addition, the OPIUM tool |25 1 uses 
PB constraints and optimizes a user provided single objective function. One modelling 
example could be preferring smaller packages to larger ones. 

The encoding of these constraints into SAT is straightforward: for each package Pi 
there is a Boolean variable Xi that is assigned to true iff package is installed, and 
clauses are either dependency clauses or conflict clauses (one clause for each pair of 
conflicting packages). 

Example 1 Given a set of package constraints S = {{pi, {p2,P5Vp6}, 0), {P2, 0, {ps}), 
(P3j {Pi}^ {Pi})^ (P4 7 01 {PSiPe})}. its encoded CNF instance is the following: 

-iXi V X2 "i^s V a;4 
-iXi y Xc^y Xq -1X3 V -^Xi 
-1X2 V -12:4 V -^X5 

-1X4 V -iXq 

The problem described above is called software installability problem. The pos- 
sibility of upgrading some of the packages (or introducing new packages) poses new 
challenges as existing packages may eventually be deleted. The goal of the software 
upgradeability problem is to find a solution that satisfies user preferences by minimiz- 
ing the impact of introducing new packages in the current system, which is a reasonable 
assumption. Such preferences may be distinguished establishing the following hierar- 
chy: (1) constraints on packages cannot be violated, (2) required packages should be 
installed, (3) packages that have been previously installed by the user should not be 
deleted, (4) the number of remaining packages installed (as a result of dependencies) 
should be minimized. 



The software upgradeability problem can be naturally encoded as a weighted partial 
MaxSAT problem. In weighted MaxS AT, each clause is a pair (C, w) where C is a CNF 
clause and w is its corresponding weight. In weighted partial MaxSAT, hard clauses 
must be satisfied, in contrast to the remaining soft clauses that should be satisfied. Hard 
clauses are associated with a weight that is greater than the sum of the weights of 
the soft clauses. A solution to the weighted partial MaxSAT problem maximizes the 
sum of the weights of the satisfied clauses. However, if the solution found exceeds the 
weight associated with a hard clause, then at least one hard clause is unsatisfied, and 
consequently the solver returns no solution. 

The following example shows a weighted partial MaxSAT formula for the upgrade- 
ability problem. 

Example 2 Given a set of package constraints S — {(pi, {P2,P5}, {^4}): {P2, 0, 0), 
(P3i {P2 Vp4}, 0), (pi, 0, 0), (p5, 0,0)}, the set of packages that the user wants to install 
I = {pi}, and the current set of installed packages in the system A = {P2}, its encoded 
weighted partial MaxSAT instance is the following: 

(-ixi V X2, 16) 
(-■a;! V X5, 16) 
(-10:1 V -^Xi, 16) 
(-1X3 V 2:2 V X4, 16) 
(xi,8) 
{X2A) 
(-2:3,1) 

In Example |2l clause weights depend on the kind of clauses we are encoding. We 
assign a hard weight (with value 16) to clauses encoding the dependencies and con- 
flicts. A maximum weight (with value 8) is assigned to the packages the user wants 
to install. A medium weight (with value 4) is assigned to clauses encoding packages 
currently installed in our system in order to minimize the number of removed packages. 
Finally, the minimum weight (with value 1) is assigned to clauses encoding the remain- 
ing packages in order to minimize the number of additional packages being installed as 
a result of dependencies. 

As we can see, this weight distribution gives priority to the user preferences over all 
the other packages, and it also gives priority to the current installation profile over the 
remaining packages. 

3 Multilevel Optimization 

The software upgradeability problem described in the previous section can be viewed 
as a special case of the more general problem of Multilevel Optimization [loQ. Multi- 
level optimization can be traced back to the early 70s [|8J, when researchers focused on 

' This problem is also referred to as Multilevel Programming f9\ and Hierarchical Optimiza- 
tion [4j. 



mathematical programs with optimization problems in the constraints. Multilevel opti- 
mization represents a hierarchy of optimization problems, where the outer optimization 
problem is subject to the outcome of each of the enclosed optimization problems in or- 
der In part motivated by the practical complexity of the multilevel optimization, most 
work in the recent past has addressed the special case of bilevel optimization ifTOl . 
Moreover, and for the special case of integer or Boolean variables, existing work is still 
preliminary [11|. It should also be observed that the general problems of bilevel and 
multilevel optimization find a wide range of applications ifTOl . representative examples 
of which can be represented with integer or Boolean variables f22]. 

One can conclude that the software upgradeability problem can be viewed as an ex- 
ample of multilevel programming, where the constraints are clauses, and the variables 
have Boolean domain. The least constrained (or outer) optimization problem represents 
the problem of minimizing the number of newly installed packages due to dependen- 
cies, whereas the most constrained (or inner) optimization problem represents the prob- 
lem of maximizing the installation of packages in the user preferences. 

This paper focuses on the special case of multilevel optimization where the con- 
straints are propositional clauses and the variables have Boolean domain. This problem 
will be referred to as Boolean Multilevel Optimization (BMO). The hierarchy of opti- 
mization problems can be captured by associating suitable weights with the clauses, as 
illustrated for the package upgradeability problem. 

More formally, consider a set of clauses C = CiUC2U---U Cm , where Ci , C2 , . . . , 
Cm form a partition of C. Moreover, consider the partition of C as a sequence of sets 
of clauses: 

{Cl,C2,...,Cm) (1) 

Where a weight is associated with each set of clauses: 

{wi,W2,-.-,Wm) (2) 

As with MaxSAT, clauses with weight Wm are required to be satisfied, and so are re- 
ferred to as hard clauses. The associated optimization problem is to satisfy clauses in 
Ci U C2 U . . . U Cm such that the sum of the weights of the satisfied clauses is maximized. 
Moreover, the hierarchy of optimization problems is captured by the condition: 

Wi > ^ Wj ■ \Cj\ i = 2,...,m (3) 

i<j<i 

The above condition ensures that the solution to the BMO problem can be split into a 
sequence of optimization problems, first solving the optimization problems for the soft 
clauses with the largest weight (i.e. Wm-i), then for the next clause weight, and so on 
until all clause weights are considered. Building on this observation, the next section 
proposes dedicated algorithms for BMO. 

4 Solving Boolean Multilevel Optimization 

This section describes alternative solutions for BMO, in addition to the weight-based 
solution described earlier in the paper The first solution is based on iteratively rescaling 
the weights of the MaxSAT formulation. The second formulation extends the standard 
encoding of weighted MaxSAT with PB constraints. 



4.1 BMO with MaxSAT 



Consider the BMO problem specified by equations ([TJ, Q and (O. 

The use of MaxSAT considers a sequence of to — 1 MaxSAT problems, at each step 
rescaling the weights of the clauses and the initial upper bound (UB) for each problem. 
The first MaxSAT problem is defined as follows: 

Initial UB: |C„_i| + 1 

A,gc„(cjan-i| + l) (4) 
Ae6C™_,(c,l) 

Let the optimum solution be u„i-i, that represents the minimum sum of weights of 
falsified clause^ In this case, as the weights of clauses to optimize is one, Um~i will be 
the minimum number of falsified clauses. The remaining MaxSAT problems can then 
be defined as follows: 



Initial UB: (u,„_i + I) ■ p> 



m — l 



AT=^+2 ^ceC, (c, (Uj-i + 1) -Pj-i) 
^ceC.J<^A\C^\ + l)■p^) 
Ac6C.(c,l) 

With 1 < i < TO — 1, where pi is the weight used for the set of clauses Ci within 
the same subproblem, and the optimum solution is Ui. Observe that the value of Pm-i 
is refined for each iteration of the algorithm. Also note that in this case, Ui represents 
the minimum number of clauses that needs to be falsified for clause weight Wi, taking 
into account that for larger weights, the number of falsified clauses must be taken into 
account. The last problem to be considered corresponds to i — 1, for the clauses with 
the smallest weight. 

Finally, the MaxSAT solution for the original problem is obtained as follows: 

m — 1 

J2 W^ ■ - U,) (6) 

i=l 

Proposition 1 The value obtained with dS]), where the different Ui values are obtained 
by the solution of the and Q MaxSAT problems, yields the correct solution to the 
BMO problem. 

Vroof: (Sketch) Proof follows from the above explanation, taking into account the con- 
dition on clauses' weights ©• ■ 



^ The MaxSAT problem is often referred as the MinUNSAT problem. 



4.2 BMO with PB 



The efficacy of the rescahng method of the previous section is still bound by the weights 
used. Even though the rescaling method is effective at reducing the weights that need 
to be considered, for very large problem instances the challenge of large clause weights 
can still be an issue. An alternative approach is described in this section, which elim- 
inates the need to handle large clauses weights. This approach is based on solving the 
BMO problem as a sequence of PB problems. 

Consider the BMO problem specified by equations ([T]l, (|2]i and Each set of 
clauses Ci can be modified by adding a relaxation variable to each clause. The resulting 
set of relaxed clauses is C^, and the set of relaxation variables used is denoted by Yi. For 
example, if cj G Ci, the resulting clause is cj^r — Cj ^Uj^ and yj G Yi. The technique of 
solving MaxSAT by using relaxation variables to clauses is a standard technique [3 2|. 

The next step is to solve a sequence of PB problems. The first PB problem is defined 

as: 

min J2yeY^^^ V 

s-t- Acec„ c (7) 

Let the optimum solution be Wm-i represents the largest number of clauses that 

can be satisfied, independently of the other clause weights. 

Moreover, the remaining PB problems can then be defined as follows: 

min E^GY, y 
s-t- Acec„. c 

Arn-lf. \ (8) 

With 1 <i < m—1, and where the optimum solutionis Ui. In this case, represents the 
largest number of clauses that can be satisfied for clause weight Wi, taking into account 
that for larger weights, the number of satisfied clauses must be taken into account. The 
last problem to be considered corresponds to i = 1, for the clauses with the smallest 
weight. 

Finally, given the definition of Vi, the PB-based BMO solution is obtained as fol- 
lows: 

m — 1 

'^Wi-Vi (9) 

i=l 

As can be concluded, the proposed PB-based approach can solve the BMO problem 
without directly manipulating any clause weights. 

Proposition 2 The value obtained with (O, where the different Vi values are obtained 
by the solution of the (0 and ^ PB problems, yields the correct solution to the BMO 
problem. 



Proof: (Sketch) Proof follows from the above explanation, taking into account the con- 
dition on clauses' weights 01). ■ 



5 Experimental Evaluation 

This section describes the experimental evaluation conducted to show the effectiveness 
of the new algorithms described above. With this purpose, we have generated a com- 
prehensive set of problem instances of the software upgradeability problem. In a first 
step, a number of off-the-shelf MaxSAT and PB solvers have been run. In a second 
step, these MaxSAT and PB solvers have been adapted to perform BMO approaches. In 
what follows we will use BMO''*^ to denote weight rescaling BMO with MaxSAT and 
BMO^P^ to denote BMO with iterative pseudo-Boolean optimization. 

5.1 Experimental Setup 

The problem instances of the upgradeability problem have been obtained from the 
Linux Debian distribution archivqj, where Debian packages are daily archived. Each 
daily archive is a repository. Two repositories corresponding to a snapshot with a time 
gap of 6 months have been selected. From the first repository, the packages for a basic 
Debian installation have been picked, jointly with a set of other packages. From the sec- 
ond repository, a set of packages to be upgraded have been picked. This set of packages 
is a subset of the installed packages. 

Each problem instance is denoted as i<x>u<y> where x is the number of installed 
packages (apart from the 826 packages of the basic installation) and y is the number of 
packages to be upgraded. In the following experiments the number of x packages ranges 
from to 2000 and the number of y packages is 98. The y packages correspond to the 
subset of packages of the basic installation that have been updated from one repository 
to the other 

The four MaxSAT solver^ used for the evaluation are: IncWMaxSatz flSl . Mini- 
MaxSat fl5|, Sat4jMaxsat [TT] and WMaxSatz |6|. The four PB solver^ used for the 
evaluation ai-e: Bsolo |21|, MinisatH- [12J, PBS4 [U and Sat4jPB 117|. Other solvers 
could have been used, even tough we believe that these ones are some of the most 
competitive and overall implement different techniques which affect performance dif- 
ferently. For each solver a set of instances were run with the default solver and BMO'^'*'^ 
orBMO*^'^ 

Furthermore, for the best performing solver an additional number of instances has 
been run in order to study the scalability of the solver as the number of packages to 
install increases. 

The experiments were performed on an Intel Xeon 5160 server (3.0GHz, 1333Mhz 
FSB, 4MB cache) running Red Hat Enterprise Linux WS 4. For Sat4j was used JRE 
1.6.0-0.07. For each instance was given the timeout of 900 seconds. 

' http://snapshot. debian . net 
http : //www . max sat . udl . cat / 
^ http : / / www . cril . univ-artois . fr/PB07/ 



5.2 Experimental Results 

Table[T]shows the CPU time required by MaxS AT solvers to solve a set of given problem 
instances. Column Default shows the results for the off-the-shelf solver and column 
BMC"^ shows the results for the weight rescaling approach specially designed for 
solving BMO problems with MaxSAT. For each instance the best result is highlighted 
in bold. 

Clearly, IncWMaxSatz with BMO'^^'^ is the best performing solver. Nonetheless, 
every other solver benefits from the use of BMO'"*''^. The only exception is Sat4jMaxsat 
because it spends around 8 seconds to read each instance and with BMC*^ the solver 
is called three times for the instances considered. Another advantage of using BMO'^^'^ 
is that the solvers do not need to deal with the large integers representing the clause 
weights, which are used in the default encoding. This can be such a serious issue that 
for some solvers there are a few problem instances (designated with '-') that cannot be 
solved. 





IncWMaxSatz 


MiniMaxSat 


Sat4jMaxsat 


WMaxSatz 
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BMO"'= 
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BMO'''"= 


iOu98 
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3.29 
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25.88 


276.32 


13.19 


i20u98 


4.72 


3.67 
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5.64 




95.89 




27.57 
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i500u98 


22.98 


4.82 




126.97 




46.51 




>900 


il000u98 


37.47 


5.74 




195.54 
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>900 


i2000u98 


45.69 


7.39 




223.81 




685.17 




>900 



Table 1. The software upgradeability problem with weighted partial MaxSAT solvers 
(time in seconds) 



Table |2] shows the results for PB solvers on solving the same instances. BMO*^''' 
boosts the solvers performance, being Sat4jPB the only exception (for some instances 
it improves, for some other it does not). For the remaining solvers, the improvements 
are significant: most of the instances aborted by the default solver are now solved with 
BMO*P''. Although there is no dominating solver, in contrast to what happens with 
IncWMaxSatz in the MaxSAT solvers, Bsolo is the only solver able to solve all the 
instances with BMO*^^. Also, despite existing a trend to increasing run times as the 
size of the instances increase, there are a few outliers. This also contrasts with MaxSAT 
solvers, but is no surprise as additional variables can degrade the solvers performance 
in an unpredictable way. 

Finally, we have further investigated IncWMaxSatz, which was the best performing 
solver. Figure [U shows the scalability of the solver comparing the default performance 
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40.54 



Table 2. The software upgradeability problem with pseudo-Boolean solvers (time in 
seconds) 



of IncWMaxSatz with its performance using BMO'^'"^. (The plot includes results for 
additional instances, with each point corresponding to the average of 100 instances.) 
We should first note that the default IncWMaxSatz solver is by far more competitive 
than any other default MaxSAT or PB solvers. Its performance is not even comparable 
with WMaxSatz, despite IncWMaxSatz being an extension of WMaxSatz. This is due 
to the features of IncWMaxSatz that make it particularly suitable for these instances, 
namely the incremental lower bound computation and the removal of inference rules 
that are particularly effective for solving random instances. Nonetheless, BMO'^'*^ has 
been able to improve its performance and to reduce the impact of the size of the instance 
in the performance. 

6 Conclusions 

In many practical applications, one often needs to solve a hierarchy of optimization 
problems, where each optimization problem is specified in terms of a sequence of nested 
optimization problems. Examples in AI include specific optimization problems with 
preferences. Another concrete example is package management in software systems, 
where SAT, PB and MaxSAT find increasing application. It is possible to relate these 
optimization problems with multilevel (or hierarchical) optimization |8 9 10 1, which 
finds a large number of practical applications. Despite the potential practical applica- 
tions, work on multilevel optimization algorithms with linear constraints and integer or 
Boolean variables is still preliminary 1 11 1. 

This paper focus on Boolean Multilevel Optimization (BMO) and, by considering 
the concrete problem of package upgradeability in software systems, shows that existing 
solutions based on either MaxSAT or PB are in general inadequate. Moreover, the paper 
proposes two different algorithms, one that uses MaxSAT and another that uses PB, to 
show that dedicated algorithms for BMO can be orders of magnitude more efficient than 
the best off-the-shelf MaxSAT and PB solvers. 




Fig. 1. Scalability of the solver IncWMaxSatz in its default version and using the 
BMC^"^ when increasing the number of packages to install from to 2000 



Despite the very promising results, a number of research directions can be outlined. 
One is to evaluate how the proposed algorithms scale for larger problem instances. 
Another is to consider other computational problems in AI that can be cast as BMO, for 
example in the area of preferences and in the area of SAT-based optimization. 
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